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ABSTRACT 

The operating concepts emerging under the Next Generation air transportation system (NextGen) require new technology 
and procedures - not only on the ground-side - but also on the flight deck. Flight deck display and decision support 
technologies are specifically targeted to overcome aircraft safety barriers that might otherwise constrain the full 
realization of NextGen. One such technology is the very lightweight, unobtrusive head-wom display (HWD). HWDs 
with an integrated head-tracking system are being researched as they offer significant potential benefit under emerging 
NextGen operational concepts. Two areas of benefit for NextGen are defined. First, the HWD may be designed to be 
equivalent to the Head-Up Display (HUD) using Virtual HUD concepts. As such, these operational credits may be 
provided to significantly more aircraft for which HUD installation is neither practical nor possible. Second, the HWD 
provides unique display capabilities, such as an unlimited field-of-regard. These capabilities may be integral to 
emerging NextGen operational concepts, eliminating safety issues which might otherwise constrain the full realization of 
NextGen. The paper details recent research results, current HWD technology limitations, and future technology 
development needed to realize HWDs as a enabling technology for NextGen. 
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1. INTRODUCTION 

Although today’s National Airspace System (NAS) is one of the safest means of transportation, it has evolved into a 
large, complex, and loosely integrated network of systems that is especially prone to delays cascading throughout the 
system because of adverse weather conditions. The Next Generation Air Transportation System (NextGen) concept for 
the year 2025 and beyond envisions the movement of large numbers of people and goods in a safe, efficient, and reliable 
manner. The goal of NextGen is to remove many of the constraints in the current air transportation system, support a 
wider range of operations, and deliver significantly increased system capacity to that of current operating levels. 
Operating concepts emerging under NextGen require new technology and procedures - not only on the ground-side - but 
also on the flight deck. One of the key elements is to create a NAS that is resilient, if not immune to the impacts of 
weather. 

National Aeronautics and Space Administration (NASA) research, development, test, and evaluation (RDT&E) of flight 
deck interface technologies is being conducted to proactively overcome aircraft safety barriers that might otherwise 
constrain the full realization of NextGen. As part of this work, specific research issues associated with the NextGen 
Terminal Maneuvering Area (TMA) are being addressed. This paper summarizes research and technology development 
being conducted by NASA and its partners investigating the use of Head-Worn Displays (HWDs) as a NextGen enabling 
technology for fixed-wing aircraft. 


2. BACKGROUND 

The idea of weather-immunity for NextGen is captured in many respects within the emerging operational concept 
entitled Equivalent Visual Operations (EVO). The goal of EVO is to create Visual Flight Rules (VFR)-like operational 
tempos and VFR-like safety, in all weather conditions. At many locations, such as Chicago O’Hare airport, when 
weather requires the use of Instrument Flight Rules (IFR) operations today, arrival throughput capacity drops from eight 
arrivals every five minutes down to five arrivals every five minutes (Vasquez-Marquez, 1991). The delays associated 
with IFR restrictions depend upon the air traffic mix and airport runway configuration, but more often, the change from 
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VFR to IFR operations can be dramatic (FAA, 1983). Because of the NAS interconnectivity, operational delays at one 
airport cascade throughout the NAS. 

EVO implies Category III operations to all runway ends (e.g., a decision height (DH) lower than 100 ft, or no decision 
height, or a runway visual range (RVR) less than 1200 ft). Landing in near zero visibility conditions is not new or novel. 
Automatic and manual landing systems are certified and operational down to 300 ft RVR. Unfortunately, the cost to 
develop, maintain, and train the crew and aircraft with these systems can be cost prohibitive. For instance, there are only 
126 Category III approaches in the NAS (Kramer et al, 2007) - well short of accommodating the 19,800 airports in the 
NAS (FAA, 2007). The 20 year life cycle costs for just the required Category Ill-type approach lighting system alone is 
more than $5.4M at each runway end (Kramer et al, 2008). Using today’s Category III landing systems and operating 
rules throughout the NAS is just not a practical solution for weather- immunity. 

Even with all-weather auto-land systems, previous research has shown that, while the capability may be available to 
take-off and land aircraft in near zero visibility, operational tempos and safety are significantly degraded, compared to 
VFR, due to limitations in surface operations - that is, taxiing and maneuvering aircraft and vehicles to/from the active 
runways and gates (Chin, 1997). Auto-land systems may provide some aspects of EVO, but other technologies are 
needed to fulfill the full promises of NextGen. 

For this reason, Synthetic / Enhanced Vision Systems (SEVS), in addition to other technologies, are being viewed as 
‘enabling NextGen’ avionics to provide ‘greater access and throughput at airports’ (JPDO, 2008). SEVS are “vision- 
based” technologies intended to create, supplement, or enhance the natural vision of the pilot (Kramer et al, 2008). 
Enhanced Flight Vision Systems (EFVS) - a certified and approved enhanced vision system used on a Head-Up Display 
(HUD) - enable a pilot to continue descent below the published DH in lieu of the pilot’s natural vision view of the 
approach and landing visual references. Synthetic vision is a computer-generated image of the external scene 
topography that is generated from aircraft attitude, high-precision navigation, and data of the terrain, obstacles, cultural 
features, and other required flight information. These two technologies are complementary and, when combined, the 
potential for all-weather operations using an electronic means of providing the pilot with an “equivalent vision 
capability” are realizable (Bailey, Kramer, Prinzel, 2005). 

The emerging vision-based technologies of SEVS and the need and emergence of new NextGen operational concepts 
may create two motivations (or market segments) for the introduction of the HWD into commercial and business 
aviation: 1) equivalent HUD capabilities; and 2) new NextGen operational capabilities. 

3. EQUIVALENT HUD CAPABILITIES 

The advantages of a HUD in commercial and business aircraft operations are many, but two hold particular appeal: 

• The conformal display of critical flight parameters/symbology increases safety by reducing the visual 
accommodation and re-accommodation that the pilot would otherwise have between reading head-down 
instrumentation and transitioning to out-the-window visual references during operations such as approach, 
landing, taxiing, and take-off. The safety benefits of the HUD were recently quantified (Flight Safety 
Foundation, 2009). The study found that HUDs prevented or positively influenced 38% of accidents overall in 
modem glass cockpits. Of those accidents where the pilot was directly involved, such as takeoff or loss-of- 
control, the “safety advantage” of the HUD became much higher (69% and 57%, respectively). 

• The explicit display of flight path and energy information on the HUD promotes precision touchdown capability 
and energy management - i.e., maximizing landing performance (minimizing aircraft wear-and-tear). Studies 
show that significantly lower touchdown sink rates are flown under manual than automatic landings 
(Rustenburg, Tipps, and Skinn, 2001) and experimental evidence show that the HUD promotes better sink rate 
control and touchdown performance (Bailey, Kramer, and Williams, 2010; Kramer et al, 2011). 


Despite these advantages, the introduction of HUDs in the commercial and business aircraft fleet took considerable time. 
HUDs first flew in limited commercial airline service in the 1970s (Spitzer, 2000) but finally, by the 1990s, the HUD 
advantages became cost-effective. One HUD manufacturer, Rockwell Collins (Flight Dynamics prior to 1998), now has 
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more than 3,300 HUDs in service worldwide, certificated on 40 air transport category aircraft, including on a variety of 
executive jets built by Bombardier, Dassault, Gulfstream and Boeing (Croft, 2010). The HUD is becoming well 
ingrained in commercial aviation. The Boeing 787 is equipped with dual HUDs for the Captain and First Officer as 
standard equipment. 

3.1 HUD “Operational Credit” 

Thanks in part to the characteristics of the HUD, and in addition to safety benefits, “operational credits” are now being 
derived from HUD equipage. 

These HUD-unique credits include: 

• Fail-passive landing capability to 50-foot DH and RVR of 600-foot using HUD-driven guidance through 
approach, flare, landing, and roll-out (as per Federal Aviation Administration (FAA) Advisory Circular 
(AC)120-28D); 

• Low visibility takeoff minima of 300 ft RVR (as per AC120-28D); 

• Special Category II minima on Type I ILS of 100-foot DH, 1,200-foot RVR (as per FAA Order 8400.13); 

• Reduction in Category II minima to 1,000-foot RVR (as per FAA Order 8400.13); and 

• Special Category I minima of 150-foot DH, 1400-foot RVR in lieu of centerline and touchdown zone lighting 
(as per FAA Order 8400.13). 

Investigation by NASA and others is also being conducted for the possibility of operational credit (lower DH or reduced 
visibility minima) from the use of Synthetic Vision displayed on the HUD (Figure 1). 

Also, the HUD is the only display currently certified and approved for use as an EFVS. With EFVS, a pilot may descend 
below the published decision altitude (DA) from a straight-in instrument approach using an EFVS (see Figure 1) in lieu 
of natural vision (as per Federal Aviation Regulation §91.175 (1) and (m)). 

The EFVS operational credit (as per §91.175 (1) and (m)) explicitly expressed that the use of a HUD was an essential 
“characteristic and feature” of the EFVS operation. However, provisions for the use an equivalent display were made. 



Figure 1. Enhanced Vision HUD (Upper Left), Synthetic Vision HUD (Upper Right) 

What constitutes an equivalent display is not explicitly defined, but by inference, the display must present “the required 
features and characteristics such that they are clearly visible to the pilot flying in his or her normal position and line of 
vision looking forward along the flight path.” A critical component of EFVS performance is the integration of the 
“visual-like” imagery with symbology where the imagery is a display of the external scene from an imaging sensor, such 
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as a Forward-Looking InfraRed or millimeter wave radar. The primary reference for maneuvering the airplane is based 
on what the pilot sees through the EFVS and the conformal HUD symbology. As such, the required external visual 
references must be continuously and distinctly visible and identifiable by the pilot. 

3.2 HWD as a HUD Replacement 

With many operational credits being provided by HUD operations, one possible avenue of HWD injection is by 
providing a HUD-equivalent capability. 

The requirements for a HWD to meet a “HUD-equivalent” capability may be derived from FAA guidance material. For 
instance, under EFVS operations, these “essential features” of the HUD or equivalent display were described as follows 
(Bailey, Kramer, and Williams, 2009): 

• The display should provide the EV image and spatially-referenced flight symbology so that they are aligned 

with and scaled to the external view (i.e., conformal rendering). 

• The display should be located so the pilot is looking forward along the flight path (i.e., looking at and through 

the imagery to the out-of-the window view) to readily enable a transition from EFVS imagery to the out-the 
window view. 

• The display should not require the pilot to scan up and down between a head down display of the image and the 

out-the-window view looking for primary flight reference information. This transition would otherwise be 
hindered by repeatedly re- focusing from one view to the other. 

These requirements suggest that an HUD-equivalent display must provide conformal imagery; therefore, the HWD must 
use head- tracking to create a “Virtual HUD” concept. The Virtual HUD concept is not new. The F-35 is working 
toward making the HWD a HUD replacement (Frey and Page, 2002). However, achieving this capability for business 
and commercial aircraft is a formidable challenge. 

For commercial and business aircraft, three potential markets are evident for a HWD HUD-equivalent. These markets 
exist where: 

• HUD installation is not possible or practical by volume or weight; 

• HUD retro-fit is not cost-effective; and, 

• HWD installation has a retum-on-investment advantage to the HUD. 

In many aircraft, a HUD installation is not possible or practical, principally from a volume standpoint. Current 
generation HUDs require a significant overhead volume for the projector. For this reason, many new aircraft and retro- 
fit applications of a HUD are impossible or impractical because the aircraft doesn’t have enough overhead space. This 
limitation is well-recognized and many avionics vendors are trying to develop new HUD technologies to overcome this 
limitation. For instance, the Micro- Vis™ by Kollsman and the Q-HUD™ by BAE Systems (Cameron, 2009) are 
adapting waveguide optics technologies to eliminate the HUD projector. The cost of the Micro- Vis HUD is also 
advertised to be an order of magnitude less expensive than a ‘traditional’ HUD, being around the $50,000 price point, 
compared to $500,000 for traditional overhead projection systems (Pope, 2006). The challenge for these new 
technologies - just like the HWD - is to meet the extremely high optical performance standards of current-day HUDs. 

The HWD also presents a much easier installation and retro-fit path than a HUD. Since, by definition, the HWD must be 
lightweight and un-encumbering to the user, the installation does not require overhead hard-points from which to mount 
combiner glass and wiring harnesses for a HUD. 

Not only does volume get into the factors influencing HWD introduction into commercial and business aviation, but the 
weight of current HUD systems is also a factor. Current overhead and combiner technologies can easily weight 25 lbs or 
more. This weight is far greater than the desired 0.25 lbs head-borne weight of the ideal HWD. The retum-on- 
investment by saving weight is obviously different for each operator, but one perspective is provided by American 
Airlines ( http://www.aa.eom/i 1 8n/amrcorp/newsroom/fuel- smart. j sp) . In their analysis, every pound removed from an 
aircraft can help save as much as 12,000 gallons of fuel annually fleet- wide. For this reason, American Airlines 
conducted significant weight-sembs, including the removal of such unnecessary cabin items as old phone equipment, 
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magazine racks and razor outlets. The possibility of saving 25 lbs for each HUD-equipped aircraft might create a 
reasonable return on investment. 

The primary technical challenge for a HUD-equivalent HWD is to create equivalent optical performance. One might 
also consider other factors such as those cited for the lack of acceptance of HWDs in consumer and military applications 
(Havig et al, 2009). In combination, these create a formidable obstacle and one which grows each day as HUD 
technology moves toward higher brightness, larger fields-of-view and greater resolutions. Where a 32 deg H x 24 deg V 
field-of-view HUD was state-of-the-art five years ago, active-matrix Liquid Crystal Display technology with light- 
emitting diodes backlighting now create a HUD that’s 20% lighter than the previous generation with 1,400 x 1,050 
pixels covering a horizontal range of 42 degrees and vertical swath of 30 degrees (Croft, 2010a). Brightness levels are 
also approaching that required for full daylight readability of SEVS imagery where the backlights are said to provide 4 
shades of green at 10,000 ft-Lamberts (Bailey, 2002). 

The potential market for HUDs (or HWD-equivalents) is big and getting bigger. HUDs are quickly becoming standard 
equipment on commercial transports, and Boeing predicts that the single-aisle fleet will double by 2029 and the twin- 
aisle market will grow even more, from 3,500 airplanes in operation now, to 8,260 in 2029 (Boeing, 2010). Business 
aviation is traditionally the proving ground for advanced safety and efficiency technologies. Forecasts suggest that after 
bottoming out in 2009-2010 from 7,500 to 11,500 light, medium, and large/ultra-large business jets will be delivered in 
the next ten years (Bombardier, 2009; AOPA, 2009). If you consider that the “core market” for this technology might be 
small and medium business jets which cannot accommodate a traditional HUD, there are an estimated 8,700 aircraft 
deliveries expected from 2009 to 2019 (Bombardier, 2009). SEVS technologies are standard equipage on the flight deck 
of today’s large/ultra-large business jets so the smaller business jets represent prime opportunity for HWD growth by 
providing HUD-equivalent and EFVS operational credits. 

4. NEW OPERATIONAL CONCEPTS AND CAPABILITIES 

HWDs for commercial and business aviation may also emerge by providing NextGen operational and safety capabilities 
that cannot be fulfilled by any other display device. 

Back in the 1990s, NASA research to develop technologies to improve aircraft surface operations efficiency and safety 
led to the development of the Taxiway-Navigation and Situation Awareness (T-NASA) system (Foyle et al, 1996) and 
the Runway Incursion Prevention System (Jones et al, 2006). These systems showed the importance of taxiway 
awareness, route awareness, and taxi guidance on HUD and head-down displays to enable taxi times, approaching those 
achievable in VFR operations, and to prevent pilot error in surface operations, including missed taxi clearances, missed 
understanding of taxi instructions, making wrong turns, and becoming lost on the airport surface. 

However, this research also noted that two of the major HUD limitations during ground operations were their 
monochrome form and limited, fixed field-of-view. 

Two ground-based simulation experiments were subsequently conducted to evaluate if a HWD could provide benefits in 
performance and safety over a HUD during current-day aircraft surface operations at Chicago, O’Hare International 
Airport. The key HWD attributes evaluated in these experiments were: 1) HUD equivalence using a virtual HUD 
concept, to that provided as the T-NASA HUD; and, 2) off-boresight enhancements using the full field-of-regard 
potential of the HWD. The results showed that HUD and HWD were essentially equivalent, and a slight improvement 
was gained for the HWD in terms of surface traffic awareness, as measured under “rare event” conditions, due to the 
unlimited field-of-regard (Arthur et al, 2009). 

Under NextGen, emerging surface operational concepts will create even more trying conditions. NextGen must 
accommodate increased traffic and surface operations can quickly become a choke-point, especially during low visibility 
conditions. Further, 4-D required time of arrivals are expected in surface routing, as part of the NextGen trajectory- 
based operations. The off-boresight, head-out capability of the HWD promises to be a key enabling technology for 4-D 
surface operations in NextGen (Bailey et al, 2011). 

NASA is conducting RDT&E of a “Better- Than-Visual” (BTV) operational capability. BTV operational capability 
builds from and expands the NextGen concept of EVO (Bailey et al, 2011). The BTV operational concept replicates the 
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capacity of today’s VFR operations and more importantly, meets and improves on the safety of today’s VFR flight in all- 
weather NextGen conditions. BTV is enabled by an electronic means of displaying intuitive visual references of the 
external world and other required flight references on flight deck displays. By providing VFR-like intuitive imagery to 
the pilot/flight crew, VFR-like operational tempos and VFR procedures are envisioned for all weather conditions. As a 
minimum, the excellent safety record of VFR operations is met and even improved upon by augmenting these displays to 
show the “invisible” safety hazards during actual VFR, such as clear air turbulence and wake vortices, and precise path 
control information to promote stabilized approach procedures. 

This BTV operational concept (Figure 2) suggests that the minimum aviation system performance standard for BTV 
technologies should be that as defined by human performance in the same operation using windows during today’s VFR 
operations. The concept implies that the HWD provides an electronically-generated view of the world, as if the pilot 
were looking outside the windows on a clear day. Significant research is required to quantify this hypothesized 
performance standard, and more importantly, to determine if this standard is indeed an appropriate and sufficient 
standard for BTV. If it is, the HWD, providing an unlimited field-of-regard, is a critical element to achieving this 
concept. 

Further, the HWD provides: 

1 . Spatially-Integrated Display Concepts : Combinations of aircraft-reference and earth-reference symbology and 

imagery will create “spatially-integrated” display concepts where the location of the displayed information 
provides cueing information for the pilot. The technique of symbology scene-linking facilitates efficient 
cognitive processing may also mitigate problems of attentional tunneling (e.g., Foyle et al, 1995). 

2. Reduced Clutter Visual Displays : A significant challenge in NextGen will be Net-Centric operations where 

each user of NextGen shares information to be better informed and make better decisions and actions. This 
wealth of information must be efficiently consumed and processed with appropriate and timely actions. 
The HWD can provide significant increased visual display area, thus, mitigating visual clutter. The entire 
three-dimensional volumetric space around the pilot can be used, as appropriate and necessary to off-load 
the visual channel. Binocular optics might also add the potential to use stereoptics for decluttering and 
more effective information presentation (Parrish, Williams, and Nold, 1994; Wickens, Todd, and Seidler, 
1989; Reising and Mazur, 1990). 
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4. TECHNICAL HURDLES AND EMERGING REQUIREMENTS FOR HWD APPLICATIONS 


The opportunities for HWD in commercial and business aviation, especially as NextGen operations emerge, seem 
significant. But two principle technical challenges are apparent: 

1 ) User encumbrance 

2) Equivalent HUD performance 

4.1 User Encumbrance 

Hard data are not available as to what level of encumbrance will be tolerable for the business and commercial aviation 
market with HMDs. In Havig et al, 2009, the authors cites equipment encumbrance as one of the barriers to HMD 
applications. 

For military applications, HWDs have typically been an “add-on” to helmets that aviators already fly with. The 
challenge was to minimize the weight growth and the center-of-gravity shift for the helmet and HWD. 

The assumption for the business and commercial aviation market has been the same - minimize the added weight and 
keep the shift in center-of-gravity minimal. The key difference is that in the business and commercial aviation, pilots 
don’t wear a helmet. The concept, shown in Figure 3, assumes attachment to a lightweight headset. However, headsets 
are not always worn either. The goal should be to create an HWD which has no more weight or encumbrance than that 
of the current commercial and business aviator’s only true equipage - sunglasses. 

NASA has made selected investments into head-tracker ( http://www.intersense.com/pages/27/74 ) and display technology 
development ( http://sbir.gsfc.nasa.gov/SBIR/abstracts/10/sbir/phasel/SBIR-10-l-Al.05-9370.html ) to miniaturize the 
HWD size and weight while striving to achieve the optical performance standards demanded for HUD-equivalent 
performance. 



Figure 3: Commercial HWD Concept 


4.2 Equivalent HUD Performance 

As a minimum, a HWD should be expected to meet (or exceed) the optical performance characteristics defined for 
HUDs using SAE AS8055, “Minimum Performance Standard for Airborne Head Up Display (HUD).” and those 
characteristics identified in FAA Advisory Circular (AC) 25-11, “Transport Category Airplane Electronic Display 
Systems” related to HUDs. Using these standards, some derived requirements for HWDs are reviewed which pose 
particular technical challenges. 
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4.2.1. Head Tracking 

To create a HUD-equivalent display levies some stringent requirements on head tracking performance for HWDs. A key 
performance aspect of the HUD from SAE AS 805 5 is that of symbol positioning accuracy. Additionally, requirements 
may be derived for boresighting and the integrity of the HWD accuracy levels. These requirements dictate static 
positioning (the combination of display optics errors and head-tracking positioning accuracy) and dynamic accuracy (i.e., 
latency). 

The HUD requirement under SAE AS8055 for symbol positioning accuracy is: a) no more than 5.0 mrad error on 
boresight; b) no more than 7.5 mrad when <10° diameter around boresight; and c) no more than 10.0 mrad when < 30° 
diameter around boresight. These allowable display accuracy errors are measured from the HUD Eye Reference Point. 

Derived requirement for HWDs to be HUD-equivalent would be: 

• The head-tracker (and display system errors) must provide a boresighted solution within 5 mrad. 

• These HUD accuracy levels must be maintained whenever the HWD is located within the HUD eye box, 
centered at the cockpit design eye reference point with ±1.5 inches laterally, ±1.0 inch vertically, and ±2.0 inch 
longitudinally. 

• Allowable display errors when outside this HWD/HUD eye box have not been identified. 

• With a HUD, no “boresighting” is required, so for HUD-equivalence, the capability to achieve these 
performance levels without a boresighting procedure is ideal (e.g., see Luo et al, 2005). 

• If a HUD combiner comes out of its detent (which degrades the HUD symbol position accuracy), a warning on 
the HUD is generated to alert the pilot. This same type of capability should annunciate to the pilot that the 
HWD is not currently aligned. 

4.2.2. Latency 

Head-tracker latency dictates the dynamic symbol /imagery positioning accuracy. 

An end-to-end latency requirement of no more than 20 msec has been proposed (e.g., Bailey et al, 2004) for Virtual 
HUD applications based on previous work (Frey and Page, 2002). However, the acceptable latency may become 
significantly smaller if dynamic stability is a driving requirement. SAE AS8055 standards suggest that this is the case. 

SAE AS8055 specifies the maximum allowable error of 7.5 mrad within 10° diameter of boresight using 2-standard 
deviation (sigma) allowances. The data from (Arthur et al, 2007; Bailey et al, 2007) showed that the 2 standard 
deviation of azimuth rate of pilot’s head movement was approximately 50 deg/sec. If we use this value and impose that 
the accuracy for equivalent-HUD performance must be less than 7.5 mrad (0.429 deg), the end-to-end system latency 
must be less than 10 msec to achieve HUD-equivalent levels of dynamic system performance (i.e., 0.429 deg/50 deg/sec 
~ 9 msec latency). 

The technical challenge is that this allowable latency is an ‘end-to-end’ requirement. A basic HWD with head tracking 
system, from end-to-end, is comprised of: 1) a near-to-eye display, 2) the head tracking system, 3) one or more 
symbology or image sources, 4) and the display/image processor. Each element and the communication between them 
contribute a portion to the total latency (see Figure 4). 

No commercial or standardized device is available to measure and quantify end-to-end latency. NASA has developed a 
prototype Head-worn display Latency Measurement Rig (HeLMR) for this purpose. 

The HeLMR apparatus consists of an anatomically correct, human head which is able to ‘wear’ available commercial 
and custom HMD systems. A camera is installed in place of the eye(s) in the correct image plane location. The head is 
mounted on a precision rotary stage that moves the head in a left-right-left ‘No-No’ fashion at a precise angular rate (see 
Figure 4, Figure 5). 

To measure the end-to-end latency, a space-stabilized symbol is rendered on the HWD along the boresight. As the head 
is slewed, the space-stabilized symbology becomes misaligned with respect to the outside reference, proportional to the 
end-to-end system latency and the headform angular rate. 




Figure 4: Typical Tracking HMD system 


On the HeLMR development system, the space-stabilized symbology is viewed against a lateral graduated, calibrated 
scale. In-situ measurements don’t require custom HWD programming to quantify latency. The head is moved and 
tracked dynamically. The high speed (300 + frames per second) video image from the eye camera shows the angular 
skew of the virtual to actual image due to the system latency. An image is captured and the angular difference of the 
virtual to actual is measured. This image is captured manually or automatically and the angular offset can be manually 
determined or automatically determined using image processing software. The end-to-end latency is calculated by 
dividing the measured angular displacement by the precise rate the rotary stage is moving (Figure 6). 



Figure 5: HeLMR System (Head, Camera, Rotary Stage, Stage Controller, DC Light Source} 


9 






Dynamic Head 

Actual scale image 

. Virtual image position skewed due to system latency 



Figure 6: Eye Camera Image Of Both Scales With A Moving Head 


A prototype Rockwell Collins ProViewTM S035-MTV21 coupled with a modified Intersense, Inc IS- 1200 tracker was 
used to render an integrated synthetic terrain and virtual HUD. The total system latency of the HWD system was 
measured to be 67 milliseconds. To identify the predominant latent elements, the end-to-end system latency was 
measured again but this time without the synthetic terrain being rendered. In this case, the latency dropped to 33 
milliseconds - indicative of the 30 Hz frame rate terrain rendering process. 

These data show that the use of vision system technology on the HWD introduces significant latency and even without it, 
more work is needed to reduce the latency to acceptable levels. Software and hardware methods to attain 10 to 20 msec 
latency are needed. 


4.2.3. Field-of-View 

In Havig et al (2009), a field-of-view requirement of 120 degrees horizontal by 50 degrees vertical was proposed for 
commercial and military applications. If possible (with no less resolution in terms of pixels per degree than provided by 
today’s traditional HUD), this HWD field-of-view should be clearly adequate. 

Most HUD requirements suggest 20 degrees (horizontal and vertical) are the minimum acceptable. Recent research in a 
vertical landing task (non-fixed wing) suggests that the 29 deg H x 22 deg V field-of-view was not sufficient (Arthur et 
al, 2010). Firm FOV requirements in commercial and business market are not available, but greater than 30 deg FOV 
looks like a certainty and 120 deg H FOV would be a desirable FOV objective. 

4.2.4. Clutter and Obscuration 

The HWD provides a unique challenge during surface operations as display clutter and obscuration becomes critical. 

In low visibility flight conditions (i.e., in visibility less than 2400 ft), the out-the-window view is essentially obscured. 
However, once in the flare and on the surface, this out-the-window visibility becomes extremely critical, no matter how 
limited it might be. The concept of zero-zero (i.e., zero visibility and zero ceiling) just does not practically occur. In 
actuality, some usable visibility is available. 

The available out-the-window visibility provides the ‘truth’ and at least one pilot is always head-out during the flare and 
during surface operations. The HUD or HWD must be designed to augment the prevailing visibility - providing 
sufficient information to enhance or enable the operation - without significantly obscuring it. Today, HUDs are certified 
only after demonstrating that they are in compliance with Part 14 of the Code of Federal Regulations §25.773, showing 
that the HUD design still gives the pilots “a sufficiently extensive, clear, and undistorted view, to enable them to safely 
perform any maneuvers within the operating limitations of the airplane, including taxiing takeoff, approach, and 
landing.” 
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For a HUD, only the forward view is affected by the HUD information. The pilot can always look around the HUD to 
see an unobscured view forward of the aircraft. Also, the use of a “de-cluttering” function is required for the express 
purpose “to enhance pilot performance” under AC25-11. Minimum aviation system performance standards for EFVS 
and SV (RTCA DO-315) includes requirement such that “a control shall be provided which permits the pilot flying to 
deactivate and reactivate the display of the EFVS image on demand without removing the pilot’s hands from the primary 
flight controls (yoke or equivalent) and thrust control.” 


Unlike the HUD, the HWD creates an unlimited field-of-regard informational capability, but it also may create additional 
clutter or obscuration above and beyond what the HUD may provide. 

Three possible methodologies are being investigated to mitigate unwarranted HWD clutter or obscuration: 

1. Automatic and manual HWD declutter : The HWD symbology and imagery may be designed with automatic 
and pilot-activated decluttering to minimize clutter or obscuration, mimicking current HUD concepts. For 
instance, HWD symbology and imagery ‘cut-outs’ are applied so whenever the pilot is looking inside the 
cockpit, the HWD symbology and imagery located within the cockpit are blanked. This creates an clear HWD 
view so the pilot can look inside with cockpit without having HWD information obscuring or cluttering the 
view. If the pilot wants to use the head-referenced HWD imagery or symbology when looking into the cockpit 
(e.g., they want to see information passing under the aircraft), a ‘dead-man’ switch is used to inhibit the HWD 
cut-out. In addition, a hands-on stick or throttle switch can clear the imagery or symbology (or both) on the 
HWD whenever the pilot wants or needs to have a clear view through the HWD. The challenge would be if this 
combination of automatic and manual declutter of the HWD would be sufficient to meet the §25.773 
requirement, as well as the functional/task requirements without increasing pilot workload. Research is needed 
to address if these requirements change for a monocular vs. biocular HWD. Biocular rivalry issues obviously 
become important as well and may create an interaction. Research has started to address some of these issues 
(Valimont, 2009). 

2. “Semi-Conformal”: Instead of positioning the HWD directly in front of the pilot’s eyes, NASA has conducted 
cursory investigation of “semi-conformal” concepts (Arthur et al, 2009) which can provide conformal 
information without outside visibility obscuration (see Figure 7). In the semi-conformal concept, the HWD is 
angled slightly above the ‘boresight’ of the pilot’s normal forward view. The pilot could look up to see the 
information, or to see the HWD information, with conformal overlay on the out-the-window view, the pilot 
must slightly rotate their head. In effect, the pilot makes a head-movement to bring the information into their 
view which, in essence, is the pilot’s ‘switch’ to modulate the presence or absence of HWD information. The 
key element is that the HWD does not nominally interfere with a pilot’s normal vision. 


HWD 



Figure 7: Illustration of 6 Semi-Conformal” Concept 
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3. “Beyond RVR” Control: The last concept being investigated attempts to create an intelligent declutter control 
(Figure 8). The HWD presentation is tailored knowing the natural visibility limitations (e.g., the prevailing 
visibility or RVR). In this case, earth-referenced data is not drawn knowing that the ‘ real-world ’ can be seen up 
to the prevailing visibility (i.e., the RVR). Beyond that range, where atmospheric conditions occlude the real- 
world, so earth-referenced data can be fully populated, such as SEVS technologies, knowing that it won’t 
obscure any of the pilot’s visibility of the real-world. Within the prevailing visibility, a very minimal set of 
screen- and aircraft-referenced data is drawn, only sufficient to meet the informational demands of the task. 


Modulation of the clutter and obscuration of the pilot’s view has the potential to be ‘show-stoppers’ for certification. 
Data are needed to support regulatory and design guidance for effective introduction of HWDs. 



Figure 8: Hypothetical “Beyond RVR” HWD Display 


4.2.5. Accommodation and Vision Correction 

A key element providing operational credit for HUD usage is derived in the pilot’s ability to use the head-up imagery 
while simultaneously looking forward along the flight path (i.e., looking at and through the imagery to the out-of-the 
window view) to readily enable a transition from EFVS imagery to the out-the window view. The transition by the pilot 
from using head-up display information to out-the-window references cannot be hindered or require re-focusing from 
one view to the other. 

When using HUDs or other virtual displays such as HWDs, accommodation and cognition are influenced by 
numerous complex and interacting ‘virtual display’ factors. Research has shown sometimes contradictory findings 
(Edgar, 2007) as to how the HUD or other virtual display positively or negatively impact accommodation and cognitive 
switching. These data must be reexamined for HWDs. These influences likely interact with the type of HWD 
decluttering mechanisms are being used. 

Further, the usability of the HWD, unlike a HUD, is more critically influenced by the user’s age and their 
attendant vision corrections/deficiencies. Whereas the optical design of the HUD can provide a fairly significant eyebox, 
focused near infinity, the HWD may have a much smaller exit pupil which may play havoc on a pilot who normally 
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needs eye glasses, including bi-focals and tri-focals. Commercial and business aircraft aviators become adept at using 
small eye movements to view head-down instruments while keeping their head up to use the HUD. This user population 
is a vastly different age demographic than the military Helmet-Mounted Display (HMD) community and its associated 
research data. Airline pilots can now fly to the age of 65 - very different than a military HMD aviator who, if they are 
lucky enough to still be flying at the end of their career, typically retire in their early 40s. 

5. CONCLUDING REMARKS 

A head-worn display concept, coupled with a head tracker, has significant potential for NextGen, both in terms of 
creating a HUD-equivalent capability for aircraft that can’t fit or afford a HUD or in terms of new operational or safety 
capabilities that might otherwise constrain the realization of NextGen. 

Top-level technical hurdles for acceptable commercial and business HWDs have been identified. Some performance 
requirements have been identified based on HUD-equivalent capabilities. Research in the coming years will hopefully 
provide evidence for the validity of these requirements to support effective regulatory and technical development 
decision-making. Light-weight display and robust tracking technologies are needed to drive the state-of-the-art toward 
meeting these goals. 
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